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SYSTEM AND METHOD FOR PROVIDING ACCESS TO DISPLAYED 

DATA 

Field of the Invention 

The present invention relates to a method for providing access to displayed 
data, and a system implementing the method. 

Background of the Invention 

An exemplary prior art hardware/software display arrangement in a computer 
system is shown in Fig. 1. A display memory 102 is connected to display circuitry 
104, which generates a display signal based on the contents of display memory 102. 
Display memory 102 contains data representing display windows controlled by 
various application programs, such as owning application window 106 and other 
application windows 108. 

Also present in the computer system is main memory 1 10, which contains a 
variety of programs and routines that are executed by the computer system. Main 
memory contains owning application 112, other applications 114, display 
management application program interfaces (API's) 116, device drivers 118 and, in 
this example, requesting application 120. Although the term application is used in 
this document to refer to application 1 12 and other applications 1 14, one of skill in 
the art would recognize that the description herein is equally applicable to any 
executing process in a computer system. 
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Owning application 112 owns owning application window 106, while the 
other applications 1 14 own the other application windows 108. Owning application 
112 may display data of any kind, such as text, images, video, etc., in owning 
application window 106, as well as other objects, such as control buttons, text entry 
boxes, sliders, etc. For example, owning application 112 may display data such as 
displayable data 115. In order to display data or other objects, owning application 
112 communicates with display memory 102 through API's 116, which provide a 
device-independent interface to the device-specific device drivers 118, which 
access and control the hardware of display memory 102 and display circuitry 104. 

Another application, such as requesting application 120 may request access 
to the data displayed in application window 106. A problem arises when the 
requested data is unobtainable, such as may occur when the data is protected from 
access, when the data is in a format that is not compatible with the data request, or 
when the data simply does not exist, as may occur if a window is not actually 
visible. Currently, an error is generated and no data is returned. 

A need arises for a technique that will determine what data should be 
supplied to an application when the application attempts to access data that is 
either not present, or not authorized to be accessed. 
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Summary of the Invention 

The present invention is a system and method for providing access to 
displayed data by determining what data should be supplied to an application 
when the application attempts to access data that is either not present, or not 
authorized to be accessed. In order to carry out the method of the present 
invention, a request for data of a first process is received from a second process. 
At the first process, a resolution to the request for data received from the second 
process is generated and the generated resolution is transmitted from the first 
process to the second process. 

In one embodiment of the present invention the requested data is protected 
data and the received resolution comprises a modified version of the protected 
data. The modified version of the data may comprise data substituted for at least a 
portion of the protected data or data augmenting the protected data. The data 
substituted for at least a portion of the protected data may comprise a pattern or a 
message indicating where the protected data would have been displayed. The data 
augmenting the protected data may comprise a copyright notice. 

In another embodiment of the present invention, the requested data is data 
that is not being displayed and the received resolution comprises the requested 
data. 

In another embodiment of the present invention, the requested data does not 
exist and the received resolution comprises a generated version of the requested 
data. 
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In another embodiment of the present invention, the requesting step 
comprises the step of transmitting information relating to the request for data to 
the first process and wherein the received resolution is based on the information 
relating to the request for data. The information relating to the request for data 
may identify a process requesting the data and the received resolution may be 
based on the identified process or the information relating to the request for data 
may indicate a use to be made of the data and the received resolution may be 
based on the indicated use. 

In another embodiment of the present invention, the requested data 
comprises image data. The received resolution may comprise image data having a 
different color depth than the requested data. The received resolution may 
comprise image data having a different pixel resolution than the requested data. 
The received resolution may comprise image data that is not the requested data. 

In another embodiment of the present invention, the requested data may 
comprise an image of text displayed in a first font and the received resolution may 
comprise an image of the text displayed in a second font. 

In another embodiment of the present invention, the step of generating the 

resolution comprises the steps of: transmitting a request for additional information 

relating to the request for data from the first process to the second process, 

receiving a response to the request for additional information from the second 

process, and generating, at the first process, a resolution to the request for data 

received from the second process based on the response to the request for 
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additional information received from the secoijd process. The received response 
to the request for additional information may comprise information identifying a 
process requesting the data. The received response to the request for additional 
information may comprise information indicating a use to be made of the data. 
5 The received response to the request for additional information may comprise 
information relating to attributes of the data. 

Brief Description of the Drawings 

The details of the present invention, both as to its structure and operation, 
10 can best be understood by referring to the accompanying drawings, in which like 
reference numbers and designations refer to like elements. 

Fig. 1 is an exemplary block diagram of a prior art display system. 
^ Fig. 2a is an exemplary block diagram of an embodiment of a display 

7, system, according to the present invention. 

0 

B 15 Fig. 2b is a flow diagram of a process which may be implemented in the 

system of Fig. 2b. 

Fig. 3 is an exemplary screen display generated by one embodiment of the 
present invention. 

Fig. 4 is an exemplary screen display generated by one embodiment of the 
20 present invention. 

Fig. 5 is an exemplary screen display generated by one embodiment of the 
present invention. 
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Fig. 6 is an exemplary screen display generated by one embodiment of the 
present invention. 

Fig. 7 is an exemplary screen display generated by one embodiment of the 
present invention. 

Fig. 8 is an exemplary screen display generated by one embodiment of the 
present invention. 

Fig. 9 is an exemplary screen display generated by one embodiment of the 
present invention. 

Fig. 10 is an exemplary screen display generated by one embodiment of the 
present invention. 

Fig. 1 1 is an exemplary screen display generated by one embodiment of the 
present invention. 

Fig. 12 is an exemplary screen display generated by one embodiment of the 
present invention. 

Fig. 13 is an exemplary screen display generated by one embodiment of the 
present invention. 

Fig. 14 is a data flow diagram of a process of a further embodiment of the 
present invention. 

Fig. 15 is a data flow diagram of an example of processing shown in Fig. 

14. 
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Detailed Description of the Invention 

An exemplary hardware/software display arrangement in a computer system 
is shown in Fig. 2a, in which may be implemented the process shown in Fig. 2b. A 
display memory 202 is connected to display circuitry 204, which generates a display 
5 signal based on the contents of display memory 202. Display memory 202 contains 
data representing display windows owned by various application programs, such as 
owning application window 206 and other application windows 208. 

Also present in the computer system is main memory 210, which contains a 
variety of programs and routines that are executed by the computer system. Main 
10 memory contains owning application 212, other applications 214, display 
q management application program interfaces (API's) 216, device drivers 218 and, in 

y3 this example, requesting application 220. Although the term application is used in 

5*7 this document to refer to owning application 212, other applications 214, and 

j=y requesting application 220, one of skill in the art would recognize that the description 

S 15 herein is equally applicable to any executing process in a computer system. 

Owning application 212 controls owning application window 206, while the 
other applications 214 control the other application windows 208. As shown in step 
250 of Fig. 2b, owning application 212 displays displayable data 222 in owning 
application window 206, within display memory 202. In order to create 
20 displayable data 222, owning application 212 communicates with display memory 
202 through API's 216, which provide a device-independent interface to the 
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device-specific device drivers 218, which access and control the hardware of 

display memory 202 and display circuitry 204. 

In step 252 of Fig. 2b, requesting application 220 requests access to 

displayable data 222. An application may attempt to access displayed data of 
5 another application in several ways. For example, requesting application 220 can 

access owning application window 206 through the display management API's 216 

using handle 224, or by directly accessing the physical display memory 202. 

When an application attempts to access displayed data of another application, such 

attempted access is detected, and, in step 254 of Fig. 2b, callback module 226 is 
10 invoked. In the embodiment of the present invention shown in Fig. 2a, display 

management API's 216 use process information 228 to detect the attempted access. 

Process information 228 is information that associates each region of display 

memory 202 with a process that owns that region at the present time. Other 

mechanisms may be used to detect the attempted access. Depending upon the 
1 5 detection mechanism, the callback module may be invoked by calling a registered 

function (in the case where an API detects the attempted access), by an interrupt, 

page fault, or other hardware generated condition, etc. 

In any case, in step 254 of Fig. 2b, the callback module 226 calls owning 

application 212 to request a resolution to the attempted access. Callback module 
20 226 may pass information about the attempted access, such as the identity of 

requesting application 220, a desired format, etc., to owning application 212. In 

step 256 of Fig. 2, owning application 212 invokes response module 230, which, 
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in step 258, determines what action to take and generates a resolution, examining 

the information about the attempted access if necessary. 

The action determined and resolution generated by response module 230 

may depend upon several factors, such as the type of data involved in the 

5 attempted access, the application attempting to access the data, etc. For example, 

if the data involved in the attempted access is protected, and the application 

attempting to access the data is not authorized to do so, then typically, response 

module 230 will generate a modified version of the protected data, which will be 

supplied to requesting application 220. For example, response module 230 may 

10 generate a pattern or a message, such as pattern 302, shown in Fig. 3, that would 

indicate where the protected data would have been displayed. Response module 

grj 230 may augment the protected data by, for example, adding a copyright notice 

^ 402, as shown in Fig. 4, or by blurring or obscuring some or all of the protected 

data, as shown in Fig. 5. In addition, several modifications may be applied 

gp 1 5 simultaneously. 

Other situations may arise. The data involved in the attempted access may 

not be present in display memory 202. For example, the data may be off screen, 

or it may not have been generated yet. In this case, response module 230 may 

return the actual data, generating it if necessary. Response module 230 may 

20 generate different versions of the requested data depending upon the application 

that is requesting it. For example, the displayed data may be a color image, such 

as image 602, shown in Fig. 6. A requesting application may request a black and 
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white image, in which case the response module would generate a black and white 
version of the requested image, such as image 702, shown in Fig. 7, and return the 
black and white image to the requesting application. Likewise, the requesting 
application may request a color version of a black and white image, in which case 
5 the owning application may provide the color version, if it is available, provide the 
black and white version in the requested color format, or may indicate that a color 
version of the image cannot be provided. 

Another example is shown in Figs. 8 and 9. The displayed image 802, 
shown in Fig. 8, is a high pixel resolution image. The owning application may 

10 provide a lower pixel resolution image 902, shown in Fig. 9, to the requesting 
application, either in response to a request for a particular resolution or a lower 
pixel resolution image, or because the owning application does not wish to provide 
the higher pixel resolution image to the requesting application. This may be 
useful, for example, where higher pixel resolution images are used for display 

15 purposes, but only lower pixel resolution images are provided without payment of 
a fee. The present invention may also be used to provide images having higher 
resolution than the displayed image, or to provide images of various resolutions. 
For example, the information sent with the request for the image could include 
payment information and the response module could determine what resolution 

20 image to provide. 
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Another example is shown in Figs. 8 and 9. The displayed image 1002 is 
shown in Fig. 10. The owning application may provide a modified or even a 
different image 1 102 to the requesting application. 

Another example is shown in Figs. 12 and 13. The displayed data 1202, 
5 shown in Fig. 12, includes text displayed in a font that is suitable for screen 
display, but which may not be suitable for other uses. The requested data in this 
example is not the text data, but is the data representing an image of the text. For 
example, if the requesting application will print the data including text, a higher 
; == resolution or anti-aliased version of the text may be desirable. Likewise, if the text 

:; p 10 is to be processed by optical character recognition (OCR) software, it may be 
2f desirable to provide a version that includes the text in a more OCR compatible 

=2 font, as in image 1302, shown in Fig. 13. 

M> In a further embodiment of the present invention, the owning application 

jr! and the requesting application may have further communications regarding the 

■ t Q 15 data to be provided, as shown in Fig. 14, The process shown in Fig. 14 may be 

considered an enhancement to step 258 of Fig. 2b. In step 1402, after the response 

module has been invoked in step 256 of Fig. 2b, the response module sends an 

extended request to the requesting application. The extended request is for 

information which must be provided to the response module in order for the 

20 response module to generate the proper resolution, or which is advantageous to be 

provided to the response module in order for the response module to generate an 

improved resolution. For example, the response module may request information 
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regarding "why" the requesting application wants the data, in the sense of the use 
the requesting application will make of the requested data. Likewise, the response 
module may request information about the identity of the requesting application or 
about format or attributes of the data desired by the requesting application. 

In step 1404, the requesting application provides the information requested 
by the response module in step 1402. In step 1406, the response module analyzes 
the information received from the requesting application to determine if any 
additional information is necessary. If additional information is necessary, the 
response module may repeat steps 1402-1406, in order to obtain the additional 
information from the requesting application. Once the response module has all the 
necessary information, or is unable to obtain any additional information, then, in 
step 1408, the response module determines what action to take and generates a 
resolution using the received information. If the response module is unable to 
obtain necessary information, an error may be generated, or alternatively, a default 
resolution or a resolution based on at least some default information may be 
generated. 

An example of the data flow that occurs in the processes shown in Figs. 2b 

and 14 is shown in Fig. 15. Requesting application 1502 requests access to data 

1504. The data request 1506 may include additional information, such as, in this 

example, an indication 1508 of the resolution desired by requesting application 

1502. As shown in Fig. 2b, the data request is routed to response module 1510. 

Response module 1510 may examine data 1504 and attributes 1512, in order to 
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determine an action to take in response to data request 1506. In this example, data 
1504 is an image and attributes 1512 are therefore image attributes. Attributes 
1512 include attributes such as the current resolution 1514, the current color depth 
1516, and the current magnification 1518 at which the image is being displayed, as 
5 well as available resolutions 1520, available color depths 1522, and available 
magnifications 1 524, at which the image data may be provided. 

In the example of Fig. 15, response module 1510 determines that three 
attributes, resolution, color depth, and magnification, may be specified for the 
image data to be provided and that only one attribute, resolution, has been 
10 specified by requesting application 1502. Response module 1510 then sends an 
extended request 1526 for additional information regarding the unspecified 
attributes to requesting application 1502. In this example, extended request 1526 
includes a request for a color depth specification 1528 and a request for a 
magnification specification 1530. Requests 1528 and 1530 may include 



Jj 15 information relating to the current settings 1516 and 1518 of these attributes 

and/or information relating to available settings 1522 and 1524 of these attributes. 

The requesting application receives extended request 1526 and may ignore 

the extended request or respond to it. In the example of Fig. 15, requesting 

application 1502 responds to extended request 1526 by sending extended response 

20 1532 to response module 1510. Extended response 1532 only includes a 

specification of color depth 1534 and not a specification of magnification. 

Response module 1510 may respond to this in several ways. Response module 
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1510 may return an error. Response module 1510 may repeat steps 1402-1406 of 
Fig. 14 and send one or more additional extended requests to requesting 
application 1502 in an attempt to obtain the unspecified information. Or, response 
module 1510 may use a default value, such as the current magnification 1518. 
5 Once response module 1510 has all necessary information, it generates resolution 
1536, which is the image data requested by requesting application 1502, modified 
as specified. 

Although specific embodiments of the present invention have been described, 
it will be understood by those of skill in the art that there are other embodiments that 
,=p 10 are equivalent to the described embodiments. Accordingly, it is to be understood 
that the invention is not to be limited by the specific illustrated embodiments, but 

w 
J= 

; ft only by the scope of the appended claims. 
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